Image processing apparatus and method

ABSTRACT

Provided are an image processing apparatus and method whereby even if shape data representing the shape of an object in a moving image undergoes a major change temporarily, a frame that is to be corrected can be detected and corrected in ideal fashion and it is possible to execute moving-image coding that is outstanding both visually and in terms of coding efficiency. In one example, a moving image composed of a plurality of frames is acquired from a image input unit ( 101 ) and the background image of the moving image is acquired by a background image generating unit ( 102 ). A background subtraction processor ( 103 ) extracts an object by comparing each of the frames constituting the moving image with the background image, and an abnormal-data discrimination unit ( 105 ) discriminates whether shape data representing the shape of the extracted object is abnormal or not. If the shape data is abnormal, a shape-data correction unit ( 106 ) corrects the shape data. An image-data correction unit ( 107 ) corrects a frame using the corrected shape data, and the corrected shape data and image data is coded by an encoder ( 104 ).

FIELD OF THE INVENTION

This invention relates to an image processing apparatus and method forextracting an object contained in a moving image and detecting anabnormal frame.

BACKGROUND OF THE INVENTION

Processing for separating and combining images on a per-object basis byutilizing digital techniques has become of interest in recent years. Inparticular, MPEG-4 coding has been standardized as an internationalstandard in the coding of moving images. MPEG-4 coding makes it possibleto perform coding/decoding object by object and offers the possibilityof various applications that have been difficult to achieve heretofore,examples being an improvement in coding efficiency, data distributionconforming to the transmission path and re-manipulation of images.

A technique referred to as the background subtraction method is knowngenerally as a method of extracting an object in the processing ofmoving images. This is a method in which points where changes occur aredetected by comparing a previously captured background image and anactual input image. The principles of this method will now be describedin simple terms.

First, let Pc(x,y) and Pb(x,y) represent a pixel value of an input imageand a pixel value of a background image, respectively, at coordinates(x,y) in the plane of an image. The difference between Pc(x,y) andPb(x,y) is calculated and the absolute value thereof is compared with acertain threshold value Th.

An example of a criterion formula is as follows:|Pc(x,y)−Pb(x,y)|≦Th   (1)

If the absolute value of the difference in Equation (1) is equal to orless than the threshold value Th, this indicates that there is no changeat the point (x,y) and, hence, it is decided that Pc is background. Onthe other hand, if the absolute value of the difference in Equation (1)is greater than the threshold value Th, then this indicates that thevalue has changed and that the point is one that should be extracted. Byperforming the above discrimination process at all points in an image,extraction of one frame is achieved.

FIG. 13 is a block diagram illustrating the configuration of aconventional system in which the background subtraction method andMPEG-4 coding are combined. An image input unit 1201 in FIG. 13 is forinputting a moving image and is exemplified by the image sensing unit ofa camera. Since the background subtraction method requires a backgroundimage for reference, the background image is generated by a backgroundimage generating unit 1202. Generating the background image by capturingone frame of an image beforehand under conditions in which an objectdoes not yet appear is the simplest method.

A background subtraction processor 1203 generates shape datarepresenting the shape of an object from the input image from the imageinput unit 1201 and the reference image from the background imagegenerating unit 1202. The image from the image input unit 1201 and theshape data from the background subtraction processor 1203 are input toan encoder 1204 frame by frame and the encoder proceeds to apply codingprocessing. The encoder 1204 will be described as one which executescoding in accordance with the MPEG-4 coding scheme.

If an object is to be coded, it is necessary to code the object shapeand position information. To accomplish this, first a rectangular areathat encompasses the object is set and the coordinates of the upper leftcorner of the rectangle and the size of the rectangular area are coded.The rectangular area is referred to as a “bounding box”. The area withinan object expressed by an image signal and shape signal is referred toas a “VOP” (Video Object Plane).

FIG. 14 is a block diagram illustrating in detail the structure of theencoder 1204, which executes VOP coding according to the prior art. Itshould be noted that the inputs to the encoder 1204 are image luminanceand color difference signals as well as the shape signal. These signalsare processed in macroblock units.

First, in an intra-mode, a DCT unit 1301 applies a discrete cosinetransform (DCT) to each block and a quantizer 1302 quantizes theresultant signal. Quantized DCT coefficients and the quantization widthare subjected to variable-length coding by a variable-length encoder1312.

In an inter-mode, on the other hand, a motion detector 1307 detectsmotion by a motion detection method a primary example of which is blockmatching with respect to another VOP that is adjacent in terms of time.A motion-vector prediction unit 1308 detects a macroblock that ispredicted to exhibit the smallest error relative to a macroblock ofinterest. A signal indicating motion toward a macroblock that ispredicted to exhibit the smallest error is a motion vector. An image towhich reference is made in order to generate the predicted macroblock isreferred to as a “reference VOP”.

A motion compensator 1306 applies motion compensation to the referenceVOP based upon the detected motion vector, thereby acquiring the optimumpredicted macroblock. Next, the difference between the next macroblockof interest and the corresponding predicted macroblock is obtained, DCTis applied to the resulting difference signal by the DCT unit 1301 andthe DCT coefficients are quantized by the quantizer 1302.

The shape data, on the other hand, is coded by a shape coding CAE unit1309. What actually undergoes CAE coding here are boundary blocks only.With regard to a block inside a VOP (all data within the block lieswithin the object) and a block outside a VOP (all data within the blocklies outside the object), only header information is sent to thevariable-length encoder 1312. A boundary block that undergoes CAE codingis processed in a manner similar to that of the image data.Specifically, in the interframe mode, the boundary block undergoesmotion detection by the motion detector 1307 and motion-vectorprediction is performed by the motion-vector prediction unit 1308. CAEcoding is applied to the difference value between the motion-compensatedshape data and the shape data of the preceding frame.

However, two problems described below arise with he backgroundsubtraction method set forth above.

The first problem involves the fact that this method presumes that thereis no change in the background image. Specifically, the problem is thatif a value in the background changes owing to a change in illuminationor the like, a stable result of extraction will not be obtained. Amethod of detecting a change in the background image by a statisticaltechnique and updating the background image appropriately has beendisclosed in the specification of Japanese Patent Application Laid-OpenNo. 7-302328 as a solution for dealing with this problem.

The second problem is how to deal with an instance in which a flash isfired in the middle of a scene or in which one object crosses in frontof another object. These instances will be described with reference tothe drawings. FIG. 15 is a diagram useful in describing shape datarepresenting the shape of an object in a case where a flash is fired inthe middle of a scene according to an example of the prior art.Reference numerals 1401 and 1402 denote frame data at a certain time andframe data at the next instant in time, respectively. The scene isilluminated by a flash in the second of these frames. Reference numeral1403 denotes frame data at the instant in time that follows the framedata 1402. It will be understood that the frame 1402 illuminated by theflash differs greatly from the other results of extraction (1401, 1403).

An instance where there is a change in background illumination, which isthe first problem mentioned above, primarily is merely a change inluminance value. In the case of a flash, however, which is the secondproblem mentioned above, hue also changes. As a consequence, accuratecorrection of background is difficult to achieve. Further, even ifaccurate shape data of an object has been obtained, the image data ofthe object itself also undergoes a major change. With a method such asMPEG-4, therefore, which uses an interframe difference, codingefficiency cannot be raised and the image appears unnatural visually.

FIG. 16 is a diagram useful in describing shape data representing theshape of an-object in a case where one object crosses in front ofanother object in an example of the prior art. Reference numeral 1501 inFIG. 16 denotes frame data immediately before one object (e.g., avehicle) crosses in front another object (a person), reference numerals1502 and 1503 denote frame data when the vehicle is crossing in front ofthe object that is the person, and reference numeral 1504 denotes framedata immediately after the vehicle has crossed in front of the person.If a second object thus happens to be extracted together with a firstobject, there is a major increase in amount of information ascribable toa major change in the shape data and image data. This leads to a majordecline in image quality. Since this case represents a phenomenon thatis entirely different from a change in background illumination, it isdifficult to deal with the background image by the updating method.

SUMMARY OF THE INVENTION

The present invention has been proposed in order to solve the problemsof the prior art and its object is to provide an image processingapparatus and method whereby even if shape data representing the shapeof an object in a moving image undergoes a major change temporarily, aframe that is to be corrected can be detected and corrected in idealfashion and it is possible to execute moving-image coding that isoutstanding both visually and in terms of coding efficiency.

According to the present invention, the foregoing object is attained byproviding an image processing apparatus comprising: moving-image inputmeans for inputting a moving image composed of a plurality of frames;background-image acquisition means for acquiring a background imagerelating to the moving image that has been input; object extractionmeans for extracting an object by comparing each of the framesconstituting the moving image with the background image; abnormal-datadiscrimination means for discriminating whether shape data representingthe shape of the extracted object is abnormal or not; shape-datacorrection means for correcting the shape data based upon result ofdiscrimination of the shape data; image-data correction means forgenerating image data, which represents the image of the object,conforming to the shape data that has been corrected by the shape-datacorrection means; and coding means for coding the shape data and theimage data.

The image processing apparatus according to the present invention issuch that the abnormal-data discrimination means discriminates thenecessity of correcting the shape data by comparing the shape data ofthe object from frame to frame of a plurality of frames that differ intime.

Further, the image processing apparatus according to the presentinvention is such that the abnormal-data discrimination means includes:first comparison means for comparing the shape data of the object usinga present frame and a preceding frame; second comparison means forcomparing the shape data of the object using the preceding frame and asucceeding frame; and decision means for deciding that correction of thepresent frame is necessary if a difference in the shape data of theobject between the present frame and the preceding frame is large and,moreover, a difference in the shape data of the object between thepreceding frame and the succeeding frame is small.

Furthermore, the image processing apparatus according to the presentinvention is such that the abnormal-data discrimination means includes:first comparison means for comparing the shape data of the object usinga present frame and a preceding frame; second comparison means forcomparing the shape data of the object using the present frame and asucceeding frame; and decision means for deciding that correction of thepresent frame is necessary if a difference in the shape data of theobject between the present frame and the preceding frame is large and,moreover, a difference in the shape data of the object between thepresent frame and the succeeding frame is large.

Furthermore, the image processing apparatus according to the presentinvention is such that the abnormal-data discrimination means includes:first comparison means for comparing the shape data of the object usinga present frame and a preceding frame; second comparison means forcomparing the shape data of the object using frames following thepresent frame and the preceding frame; and decision means for decidingthat correction of prescribed frames from the present frame onward isnecessary in a case where a difference between the present frame and thepreceding frame is large and, moreover, a difference in the shape dataof the object between the preceding frame and the frames following thepresent frame is small.

Further, the image processing apparatus according to the presentinvention is such that the first or second comparison means performs thecomparison using a comparison of any of the area, perimeter, waveletidentifier, circularity, centroid or moment of the shape data, or acombination thereof.

Further, the image processing apparatus according to the presentinvention is such that the shape-data correction means corrects theshape data of the object in an Nth frame, which has been determined tobe abnormal, using an (N−1)th or earlier frame determined to be normaland an (N+1)th or later frame determined to be normal.

Further, the image processing apparatus according to the presentinvention is such that the shape-data correction means further includesdetection means for detecting corresponding points in the shape databetween an (N−1)th or earlier frame and an (N+1)th or later frame.

Further, the image processing apparatus according to the presentinvention is such that the detection means obtains amount of motion ofthe corresponding points by performing pattern matching with respect toone or each of a plurality of prescribed areas.

Further, the image processing apparatus according to the presentinvention is such that the amount of motion of the corresponding pointsis amount of movement and amount of rotation of the position of thecorresponding points obtained by an affine transformation.

Further, the image processing apparatus according to the presentinvention is such that the shape-data correction means replaces shapedata in an Nth frame determined to be abnormal with shape data of an(N−1)th or earlier frame determined to be normal.

Further, the image processing apparatus according to the presentinvention is such that the image-data correction means generates imagedata representing the image of the object from a frame different from apresent frame in terms of time.

Further, the image processing apparatus according to the presentinvention is such that the image-data correction means generates an Nthframe determined to be abnormal from an (N−1)th frame determined to benormal and an (N+1)th frame determined to be normal.

Further, the image processing apparatus according to the presentinvention is such that the image-data correction means replaces an Nthframe determined to be abnormal with an (N−1)th frame determined to benormal.

Further, the image processing apparatus according to the presentinvention is such that the coding means is coding means compliant withan MPEG-4 visual coding scheme.

Other features and advantages of the present invention will be apparentfrom the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate an embodiment of the inventionand, together with the description, serve to explain the principles ofthe invention.

FIG. 1 is a block diagram illustrating the structure of an imageprocessing apparatus according to an embodiment of the presentinvention;

FIG. 2 is a flowchart for describing the operation of the imageprocessing apparatus according to this embodiment;

FIG. 3 is a flowchart useful in describing processing for detecting anabnormal frame, in which a flash has been fired, in an abnormal-datadiscrimination unit according to a first embodiment of the presentinvention;

FIG. 4 is a flowchart useful in describing another example ofdiscrimination processing for detecting a frame, in which a flash hasbeen fired, in the abnormal-data discrimination unit according to thefirst embodiment;

FIG. 5 is a flowchart useful in describing processing for correctingshape data and image data in the first embodiment;

FIG. 6 is another flowchart relating to correction of shape data andimage data in a third embodiment of the present invention;

FIG. 7 is a diagram useful in describing shape data in the firstembodiment;

FIG. 8 is a diagram useful in describing result of detectingcorresponding points in shape data in the first embodiment;

FIG. 9 is a flowchart for describing operation of the abnormal-datadiscrimination unit according to a second embodiment of the presentinvention;

FIG. 10 is another flowchart useful in describing processing forcorrecting shape data and image data in the second embodiment;

FIG. 11 is a diagram useful in describing shape data in the secondembodiment;

FIG. 12 is a diagram useful in describing processing for detectingcorresponding points in shape data in the second embodiment;

FIG. 13 is a block diagram illustrating the configuration of aconventional system in which the background subtraction method andMPEG-4 coding are combined;

FIG. 14 is a block diagram illustrating in detail the structure of anencoder for performing VOP coding according to the prior art;

FIG. 15 is a diagram useful in describing shape data representing theshape of an object in a case where a flash is fired in the middle of ascene according to an example of the prior art; and

FIG. 16 is a diagram useful in describing shape data representing theshape of an object in a case where one object crosses in front ofanother object in an example of the prior art.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Preferred embodiments of the present invention will now be described indetail with reference to the drawings.

FIG. 1 is a block diagram illustrating the structure of an imageprocessing apparatus according to a preferred embodiment of the presentinvention.

In FIG. 1, an image input unit 101, background image generating unit102, background subtraction processor 103 and encoder 104 have functionsidentical with those of the image input unit 1201, background imagegenerating unit 1202, background subtraction processor 1203 and encoder1204, respectively, described above with reference to FIG. 13. Thus,this embodiment is characterized by the fact that the encoder 104executes coding processing that is compliant with the MPEG-4 visualcoding scheme.

The image processing apparatus according to this embodiment furthercomprises an abnormal-data discrimination unit 105 for detecting anabnormality by examining shape data from the background subtractionprocessor 103; a shape-data correction unit 106 for correcting the shapedata in a case where an abnormality has been detected by theabnormal-data discrimination unit 105; and an image-data correction unit107 for correcting image data in accordance with the shape datacorrected by the shape-data correction unit 106. The altered image dataand shape data is input to the encoder 104 and is coded thereby.

FIG. 2 is a flowchart for describing the operation of the imageprocessing apparatus according to this embodiment.

As shown in FIG. 2, a moving image is input from the image input unit101 (step S101) and a background image (reference image) is generated bythe background image generating unit 102 (step S102). The backgroundsubtraction processor 103 generates shape data representing the shape ofan object from the input image that enters from the image input unit 101and the reference image that enters from the background image generatingunit 102 (step S103). The abnormal-data discrimination unit 105discriminates an abnormal frame (step S104) and the shape-datacorrection unit 106 and image-data correction unit 107 correct the shapedata of the object and the image data of the object, respectively (stepS105). The encoder 104 codes the image data and shape data after thecorrection thereof (step S106).

More specifically, the image processing apparatus according to thisembodiment acquires a moving image, which is composed of a plurality offrames, from the image input unit 101, acquires a background imagerelating to the moving image using the background image generating unit102, extracts an object by comparing the background image with each ofthe frames constituting the moving image using the backgroundsubtraction processor 103, and determines whether the shape data of theextracted object is abnormal using the abnormal-data discrimination unit105. If the shape data is determined to be abnormal, the shape data iscorrected by the shape-data correction unit 106, the frame is correctedby the image-data correction unit 107 using the shape data corrected bythe shape-data correction unit 106, and the corrected shape data andimage data is coded by the encoder 104.

Discrimination of an abnormal frame at step S104 and processing forcorrecting the shape data and image data at step S105 will be describedin detail in application to two embodiments set forth below. It shouldbe noted that processing other than that set forth below is as describedabove.

First Embodiment

An image processing apparatus according to a first embodiment of thepresent invention will be described with reference to FIGS. 3 to 8. Thisis processing for dealing with the problem (firing of a flash in themiddle of a scene) described above with reference to FIG. 15.

FIG. 3 is a flowchart for describing processing for detecting anabnormal frame, in which a flash has been fired, by the abnormal-datadiscrimination unit 105 according to the first embodiment.

First, the abnormal-data discrimination unit 105 generates shape data ofthe present frame (step S201). This is data that is the result ofextraction using a background subtraction method similar to that of theprior art. Next, the shape data of the preceding frame is compared withthe shape data of the present frame (step S202).

Several methods are conceivable as methods of comparing the shape data.A comparison of areas is one of the simplest examples of comparison. Byway of example, the number of pixels in shape data regarded as an objectmakes up the area of the object. Further, it is possible to adoptperimeter as a parameter, and expression by a curve using a Fourierdescriptor also is possible. For a detailed explanation, see “ImageProcessing Engineering; Introductory Edition” edited by YoshiharuTaniguchi. Furthermore, in this embodiment, any parameter may beselected for the comparison of shape data and there is no limitationwhatsoever upon the comparison method. In other words, this embodimentis characterized in that the method of comparing shape data may be acomparison of any of the area, perimeter, wavelet identifier,circularity, centroid or moment of the shape data, or a combinationthereof.

Processing branches depending upon the result of comparison (step S203).If the result of the comparison is that the difference is small (“NO” atstep S203), it is determined that there is no abnormality. If the resultof the comparison is that the difference is large (“YES” at step S203),then the shape data is examined. For example, if we let referencenumerals 1401 and 1402 represent shape data of the preceding frame andshape data of the present frame, respectively, then the differencebetween the frames will be large and, hence, the shape data 1402 isexamined to determine whether this frame is an abnormal frame.

As for the examination procedure, first the shape data of the succeedingframe is generated (step S204). Next, the shape data of the precedingframe is compared with the shape data of the succeeding frame (stepS205). The method of comparison in this case may be the same as thatcarried out at step S202. Processing branches depending upon the resultof this comparison (step S206).

If the decision rendered at step S206 is that the difference is small(“YES” at step S206), then it is judged that the shape data of thepresent frame is abnormal. If the difference is large (“NO” at stepS206), on the other hand, this means that the change is not a momentarychange and therefore it is judged that a flash is not the cause of thechange. In FIG. 15, since the difference between the data 1401 of thepreceding frame and data 1403 of the succeeding frame is small, it isjudged that frame 1402 is one in which the flash was fired (i.e., thatthis is an abnormal frame). If an abnormality is determined, the data ofthe present frame (the shape data and image data of the present frame)is corrected (generated again) (step S207).

More specifically, the image processing apparatus according to thisembodiment is characterized in that the abnormal-data discriminationunit 105 discriminates the necessity of the shape data correction bycomparing the shape data of an object from frame to frame of a pluralityof frames that differ in time.

Further, the image processing apparatus according to this embodiment ischaracterized in that the abnormal-data discrimination unit 105 comparesthe shape data of the object using the present frame and the precedingframe (step S202), compares the shape data of the object using thepreceding frame and the succeeding frame (step S205), and decides thatcorrection of the present frame is necessary if the difference in theshape data of the object between the present frame and the precedingframe is large and, moreover, the difference in the shape data of theobject between the preceding frame and the succeeding frame is small.

The correction processing executed at step S207 will be described ingreater detail later.

FIG. 4 is a flowchart useful in describing another example ofdiscrimination processing for detecting a frame, in which a flash hasbeen fired, in the abnormal-data discrimination unit 105 according tothe first embodiment. In the flowchart of FIG. 4, processing steps otherthan steps S208, S209 are processing steps identical with those shown inFIG. 3.

The shape data of the present frame is compared with the shape data ofthe succeeding frame at step S205. Further, if the decision rendered atstep S209 is that the difference is large, this means that a flash hasbeen fired; if the difference is small, this means that the flash hasnot been fired. If the difference is small, it can be considered thatthe state of the object has undergone a transition and that the objecthas attained a new state. For example, in FIG. 15, the differencebetween the shape data 1402 of the present frame and the shape data 1403of the succeeding frame is large, it is judged that frame 1402 is one inwhich the flash was fired.

Further, in a special case in which a sudden change in shape data cannotbe conceived as being ascribable to a cause other than a flash, it ispossible to proceed to step S207 based solely on the result of thedecision at step S203 without using the data of the succeeding frame.

In other words, the image processing apparatus according to thisembodiment is characterized in that the abnormal-data discriminationunit 105 compares the shape data of the object using a present frame anda preceding frame (step S202), compares the shape data of the objectusing the present frame and the succeeding frame (step S208), anddecides that correction of the present frame is necessary if thedifference in the shape data of the object between the present frame andthe preceding frame is large and, moreover, the difference in the shapedata of the object between the present frame and the succeeding frame issmall.

The processing (step S207) of a frame judged to require correction willnow be described.

FIG. 5 is a flowchart useful in describing processing for correctingshape data and image data in the first embodiment of the invention. Ifan object to be extracted is a rigid body, the entire body will move asone. If this is not the case, however, portions that exhibit differentmotion will appear locally. In the description that follows, an area inwhich all movements coincide shall be referred to as a “common area”,and areas other than these in which local motion occurs shall bereferred to as “motion areas”.

First, the shape-data correction unit 106 detects a common area fromshape data of the preceding and succeeding frames (step S301). This canbe found in simple fashion by performing an AND operation between theshape data of the preceding frame and the shape data of the succeedingframe. In FIG. 7, reference numerals 501 denote shape data of apreceding frame, 502 shape data of a succeeding frame, and 503 a commonarea detected from these two frames.

Further, the image-data correction unit 107 generates a common area ofimage data in the present frame (step S302). This is generated byfinding the average value between the image data of the preceding frameand the image data of the succeeding frame. That is, let Pp(x,y) andPb(x,y) represent a pixel value of a certain point (x,y) in thepreceding frame and a pixel value of the point in the succeeding frame,respectively. The average value can then be found by the calculation[Pp(x,y)+Pb(x,y)]/2.

Meanwhile, the shape-data correction unit 106 detects a motion area ofthe shape data in the preceding frame (step S303). This can be found byperforming an exclusive-OR operation between the shape data 501 of thepreceding frame and the common areas 503 of the preceding and succeedingframes. In FIG. 7, reference numeral 504 denotes the detected motionarea of the preceding frame.

The image-data correction unit 107 detects a motion area of the imagedata in the preceding frame (step S304). This may be achieved merely bycorrelating the image data with the position of the shape data of theextracted motion area. The shape-data correction unit 106 similarlyobtains a motion area of the shape data in the succeeding frame (step305). In this case also the motion area can be found y performing anexclusive-OR operation between the shape data 502 of the succeedingframe and the common areas 503 of the preceding and succeeding frames.In FIG. 7, reference numeral 505 denotes the result of detection. Theimage-data correction unit 107 then detects a motion area of the imagedata in the succeeding frame (step S306). This also may be achievedmerely by correlating the image data with the position of the shape dataof the extracted motion area.

Next, the shape-data correction unit 106 detects movement of the motionarea of shape data between the preceding and succeeding frames (stepS307). Pattern matching using an affine transformation or the like canbe employed in detecting corresponding points. An example of an affinetransformation is as follows:X=(x−x0)cos θ−(y−y0)sin θ+x0Y=(x−x0)sin θ+(y−y0)cos θ+y0   (2)where (x0,y0) represents the center of rotation and θ the angle ofrotation. Pattern matching adopts the sum total of the differencesbetween the values obtained by this calculation and the actual values orthe sum of the squares of these differences as an evaluation value andobtains the center position of rotation and the angle of rotation thatwill minimize this value.

FIG. 8 is a diagram useful in describing result of detectingcorresponding points in shape data according to the first embodiment. Inorder to simplify the description, the shape will be described as a linesegment. In FIG. 8, let line segments 601 and 603 represent data in thepreceding and succeeding frames, respectively. In this case, the angledefined by the line segments 601 and 603 is the angle of rotation in theaffine transformation, and the position of the point at which A0 and A2overlap is the center of rotation. Furthermore, points corresponding toB0 and C0 are B2 and C2, respectively.

Next, the shape-data correction unit 106 calculates motion of the shapedata of the present frame (step S308). In a case where motion isobtained at the rotation angle and center of rotation in the mannershown in FIG. 8, the amount of motion can be found by making therotation angle and center of rotation θ/2 and A0, respectively.

Furthermore, the shape-data correction unit 106 generates a motion areaof shape data in the present frame (step S309). That is, thecorresponding points are found by substituting the parameter calculatedat step S308. As a result, a point corresponding to A0 and A2 becomesA1, a point corresponding to B0 and B2 becomes B1, and a pointcorresponding to C0 and C2 becomes C1. A line segment 602 obtained byconnecting all of the corresponding points constitutes shape data of thepresent frame. Reference numeral 506 in FIG. 7 denotes shape data of themotion area of the present frame obtained in this fashion.

More specifically, this embodiment is characterized in that theshape-data correction unit 106 corrects the shape data of the object inan Nth frame, which has been determined to be abnormal, using an (N−1)thor earlier frame determined to be normal and an (N+1)th or later framedetermined to be normal.

Further, this embodiment is characterized in that the shape-datacorrection unit 106 detects corresponding points in the shape databetween an (N−1)th or earlier frame and an (N+1)th or later frame.Furthermore, this embodiment is characterized in that the shape-datacorrection unit 106 obtains amount of motion of corresponding points byperforming pattern matching with respect to one or each of a pluralityof prescribed areas. Furthermore, this embodiment is characterized inthat the amount of motion of the corresponding points is amount ofmovement and amount of rotation of the positions of the correspondingpoints obtained by an affine transformation.

Next, the image-data correction unit 107 generates image datacorresponding to the shape data generated at step S309 (step S310). Thatis, utilizing the parameter obtained at step S308, the image-datacorrection unit 107 obtains pixel values from the average values ofcorresponding points, as by obtaining the pixel value of A1 from theaverage value of A0 and A2 and the pixel value of B1 from the averagevalue of B0 and B2. It should be noted that this method of obtainingaverage values is similar to that of the procedure described at stepS302.

The shape-data correction unit 106 then combines the shape data of thecommon area obtained at step S301 and the shape data of the motion areaobtained at step S309 and re-generates the shape data of the presentframe (step S311). Reference numeral 507 in FIG. 7 denotes shape data ofthe present frame thus obtained. Furthermore, the image-data correctionunit 107 combines the image data of the common area obtained at stepS302 and the image data of the motion area obtained at step S310 andre-generates the image data of the present frame (step S312).

When the shape data and image data of the present frame is re-generatedthrough the procedure described above, processing is completed. Thisdata is then input to the encoder 104 and subjected to MPEG-4 coding inthe manner described above.

Thus, in accordance with the image processing apparatus according to thefirst embodiment, as set forth above, a frame in which shape data isdiscontinuous, as in the case of an image illuminated by a flash, isdetected, and this frame is corrected based upon the frames before andafter it. As a result, the occurrence of excessive amount of code can besuppressed with regard to both the shape data and image data and it ispossible to realize a coding system that provides excellent visualresults.

Second Embodiment

A second embodiment of the present invention will be described withreference to FIGS. 9 to 12. This embodiment relates to processing fordealing with the problem (instances where one object that is not to beextracted crosses in front of another object that is to be extracted)described above with reference to FIG. 16.

FIG. 9 is a flowchart for describing operation of the abnormal-datadiscrimination unit 105 according to a second embodiment of the presentinvention. The processing of steps S701 to S703 is the same as that ofsteps S201 to S203 in FIG. 3 described in connection with the firstembodiment. Here, however, the present frame is described as the Nthframe. Further, reference numerals 1501 to 1504 in FIG. 16 shall denoteshape data of (N−1)th, Nth, (N+1)th and (N+2)th frames, respectively. Inthis example, the shape data 1502 of the present frame is compared withthe shape data of the preceding frame and the difference is large (“YES”at step S703). The shape data, therefore, is examined.

If an object that is not to be extracted crosses in front of an objectthat is to be extracted, it is expected that a large change in shapewill be detected over a plurality of frames. The reason for this is thatsuch a change does not appear in only one frame. Accordingly, a counterm is reset to 1 (step S704) as initialization processing for counting upthe number of frames.

Next, the abnormal-data discrimination unit 105 generates shape data ofthe (N+1)th frame (the frame that follows the present frame) (stepS705). The abnormal-data discrimination unit 105 then compares thegenerated shape data of the (N+1)th frame and the shape data of the(N−1)th frame (step S706). The method of comparison is the same as thatof step S703. The abnormal-data discrimination unit 105 then determineswhether the difference between the items of data is small or not (stepS707). If the decision rendered is that the difference is small (“YES”at step S707), then it is judged that the shape data of the presentframe is abnormal. If the difference is large (“NO” at step S707), onthe other hand, then it is judged that the next frame also is to beexamined. In the example shown in FIG. 16, the shape data 1501 iscompared with the shape data 1503 and the difference is judged to belarge.

The abnormal-data discrimination unit 105 determines whether the countervalue m has attained a maximum value max (step S708). To decide themaximum value max, maximum time believed to be necessary for an objectto cross in front of another object that is not to be extracted is setbeforehand. For example, if the frame rate is 15 fps and a maximum oftwo seconds is considered, then max is set to 30. If the counter value mhas not reached the maximum value m (“NO” at step S708), then thecounter value m is incremented (step S709).

If it is found at step S708 that the counter value m has reached themaximum value max (“YES” at step S708), then error processing isexecuted (step S710). In this case, it is considered not that somethinghas crossed in front of the object to be extracted but that the shape ofthis object itself has changed in a major way. Further, in a case whereshape data that is not corrected is used as is without any problems, noparticular error processing is executed and the processing of this framemay be terminated.

The abnormal-data discrimination unit 105 repeats the processing ofsteps S705 to S709 above and exits this loop if it determines at stepS707 that the difference is small (“YES” at step S707). The countervalue m prevailing at this time will be the number of frames in whichthe result of extraction has been deemed to be abnormal. This is thenumber of frames that require re-generation of data. In the example ofFIG. 16, the shape data of frame 1501 is compared with the shape data offrame 1504 and the loop is exited at the moment m becomes 2.

At steps S711 to S714, processing for correcting (re-generating) data inthe detected m-number of frames is executed. First, initializationprocessing for setting a frame-number counter k to zero is executed(step S711). Step S712 is the main part of correction processing and thedetails thereof will be described later. Until end is discriminated(step S713) and k becomes equal to m−1, the value of k is incremented atstep S714 and the above processing is repeated. When data re-generationfrom the Nth frame to the (N+m−1)th frame ends, one series of processingsteps ends. In the example of FIG. 16, m is equal to 2 and the data offrames 1502 and 1503 is re-generated.

In the flowchart of FIG. 9, an example in which the Nth frame isprocessed is described. However, in a case where data correction hasbeen carried out, the frame processed next may be made the (N+m+1)thframe rather than the (N+1)th frame.

In other words, the image processing apparatus according to thisembodiment is characterized in that the abnormal-data discriminationunit 105 compares the shape data of the object using a present frame anda preceding frame (step S702), compares the shape data of the objectusing frames following the present frame and the preceding frame (stepS706), and decides that correction of a prescribed frame from thepresent frame or later is necessary if the difference between thepresent frame and the preceding frame is large and, moreover, adifference in the shape data of the object between the preceding frameand the frames following the present frame is small.

Processing for correcting a frame judged to require correction will nowbe described.

FIG. 10 is another flowchart useful in describing processing forcorrecting shape data and image data according to the second embodiment.

First, the shape-data correction unit 106 detects a common area fromshape data of the (N−1)th and (N+m)th frames (step S801). This can befound in simple fashion by performing an AND operation between the shapedata of the (N−1)th frame and the shape data of the (N+m)th frame in amanner similar to that described above with reference to FIG. 5. Unlikethe case of FIG. 5, however, it is considered that the common area hasmoved if the time interval between frames is long. In this case, it isso arranged that motion of the overall object to be extracted is foundby pattern matching. If the affine transformation described in FIG. 5 isused, the amount of translational motion and amount of rotation of theobject between frames can be obtained.

Specifically, if we let θ represent the angle of rotation from the(N−1)th frame to the (N+m)th frame and let (X0,y0) represent the amountof movement between these frames, then the angle of rotation in the(N+k)th frame will be θ×(k+1)/(m+1) and the amount of movement will be[x0×(k+1)/(m+1), y0×(k+1)/(m+1)]. After the positions are made tocoincide by these parameters, the common area can be detected byperforming the AND operation.

FIG. 11 is a diagram useful in describing shape data according to thesecond embodiment. In the example of FIG. 11, reference numerals 1001and 1002 denote shape data in (N−1)th and (N+2)th frames, respectively.Since there is no movement of the common area, the angle of rotation iszero and the amount of movement is (0,0). Reference numeral 103 denotesa common area obtained from the shape data 1001 and 1002.

If there is local motion outside of the common area, then detection ofthe motion area is performed. First, the motion area of the shape datain the (N−1)th frame is detected (step S802). This can be found byperforming an exclusive-OR operation after the positions of the shapedata 1001 of the (N−1)th frame and common area 1003 are made tocoincide. Reference numeral 1004 denotes the extracted motion area ofthe (N−1)th frame. Next, the motion area of the image data in the(N−1)th frame is detected (step S803). This is image data thatcorresponds to the area found at step S802.

Similarly, the motion area of the shape data in the (N+m)th frame isdetected (step S804). This can be found by performing an exclusive-ORoperation after the positions of the shape data 1002 of the (N+m)thframe and common area 1003 are made to coincide. Reference numeral 1005denotes the detected motion area of the (N−1)th frame. Next, the motionarea of the image data in the (N+m)th frame is detected (step S805).This is image data that corresponds to the area found at step S804.

Next, movement of the motion area of the shape data between the (N−1)thframe and the (N+m)th frame is detected (step S806). This involvesdetecting corresponding points between the shape data 1004 of the(N−1)th frame and the shape data 1005 of the (N+m)th frame. Detection ofthe corresponding points employs pattern matching using an affinetransformation or the like.

FIG. 12 is a diagram useful in describing processing for detectingcorresponding points in shape data in the second embodiment. In order tosimplify the description, the shape will be described as a line segment.In FIG. 12, let reference numeral 1101 denote the (N−1)th frame and leta line segment 1104 represent data in the (N+m)th frame. The angledefined by the line segments 1101 and 1104 is an angle of rotation θ′ inthis affine transformation, and the position of the point at which A0and A3 overlap is the center (x′0,y′0) of rotation. The pointscorresponding to B0 and C0 are B3 and C3, respectively.

The processing of each frame to be corrected is executed in the stepsthat follow. First, initialization processing is executed (step S807).Here the counter k for counting the number of frames to be corrected isset to zero. Next, motion of the common area in the (N+k)th frame iscalculated (step S808). This is calculated from the amount of movementobtained at step S801. In the example of FIG. 11, the angle of rotationis zero and the amount of movement is (0,0). The shape data of thecommon area is generated from the values obtained at step S808 (stepS809). In the example of FIG. 11, the amount of movement of the commonarea is zero and therefore the common area of the shape data in the Nthframe coincides with the common area 1003.

Next, image data corresponding to the shape data obtained at step S809is generated (step S810). Let P0(x,y) and Pm(x,y) represent a pixelvalue of a certain point (x,y) in the (N−1)th frame and a pixel value ofthe point in the (N+m)th frame, respectively. The pixel value in the(N+k)th frame can then be found by the calculation[P0(x,y)×(m−k)+Pm(x,y)×(k+1)]/(m+1). In a case where it is consideredthat there is almost no change in the common area, a simple method thatmay be adopted is to use the average value [P0(x,y)+Pm(x,y)]/2. An evensimpler approach that is conceivable is to utilize P0(x,y) as is. Thisis equivalent to copying the image data of the (N−1)th frame as is.

Next, movement of the motion area of the (N+k)th frame is calculated(step S811). This is found from the detection of the previously obtainedmovement between the motion area in the (N−1)th frame and the motionarea in the (N+m)th frame. Since the angle of rotation from the (N−1)thframe to the (N+m)th frame is θ′ and the amount of movement is(x′0,y′0), the angle of rotation in the (N+k)th frame is θ′×(k+1)(m+1)and the amount of movement is [x′0×(k+1)/(m+1), y′0×(k+1)/(m+1)]. Whenm=2 holds, the angle of rotation in the Nth frame is θ′/3 and the amountof movement is (x′0/3, y′0/3).

Shape data of the motion area is generated from the values obtained atstep S811 (step S812). In FIG. 12, line segment 1102 is obtained ifpoints corresponding to the rotation angle θ′/3 and amount of movement(x′0/3, y′0/3) are found with respect to line segment 1101. Next, imagedata corresponding to the shape data of the motion area is generated(step S813). The method is the same as that used to calculate the pixelvalues in the common area. If we let P′0(x′,y′) and P′m(x′,y′) representa pixel value of a certain point (x′,y′) in the (N−1)th frame and apixel value of the point in the (N+m)th frame, respectively, then thepixel value in the (N+k)th frame will be found by the calculation[P′0(x′,y′)×(m−k)+P′m(x′,y′)×(k+1)]/(m+1).

The shape data of the (N+k)th frame is re-generated (step S814). This isachieved by combining the shape data of the common area found at stepS809 and the shape data of the motion area found at step S812. Referencenumeral 1008 in FIG. 1 represents shape data that has been re-generatedin the Nth frame. Next, the image data of the (N+k)th frame isre-generated (step S815). This is achieved by combining the image dataof the common area found at step S810 and the image data of the motionarea found at step S813.

The number of frames is discriminated (step S816). If the number offrames is less than m−1 (“NO” at step S816), then the value of k isincremented at step S817 and the processing of steps S808 onward isrepeated. If m is equal to 2, then the data of the Nth and (N+1)thframes is re-generated and this series of processing steps isterminated. In the (N+1)th frame, the angle of rotation is θ′×2/3 andthe amount of movement is (x′0×2/3, y′0×2/3). If the correspondingpoints are found, line segment 1103 in FIG. 12 is obtained. Further,reference numeral 1007 in FIG. 11 denotes the motion area of the shapedata in the (N+1)th frame, and reference numeral 1009 denotes the shapearea that has been regenerated in the (N+1)th frame.

Here only one type of local motion outside of the motion area has beendescribed. However, if a plurality of motions exist, it will suffice torepeat processing for motion-area detection a number of times equal tothe number of motions.

Thus, in accordance with the second embodiment as described above, aplurality of frames in which shape data is discontinuous, as in the caseof an image obtained when an object that is not to be extracted crossesin front of an object that is to be extracted, are detected, and theseframes are corrected based upon the frames before and after them. As aresult, the occurrence of excessive amount of code can be suppressedwith regard to both the shape data and image data and it is possible torealize a coding system that provides excellent visual results.

Third Embodiment

FIG. 6 is another flowchart relating to correction of shape data andimage data in a third embodiment of the present invention. Specifically,according to this embodiment, processing for correcting the shape dataand image data by the shape-data correction unit 106 and image-datacorrection unit 107 in the first and second embodiments set forth aboveis made processing for replacing this data with shape data and imagedata of a frame determined to be normal (i.e., by copying the shape dataand image data of the normal frame).

More specifically, this embodiment is characterized in that theshape-data correction unit 106 replaces shape data in an Nth framedetermined to be abnormal with shape data in an (N−1)th or earlier framedetermined to be normal (step S401). Further, this embodiment ischaracterized in that the image-data correction unit 107 generates anNth frame determined to be abnormal from an (N−1)th frame determined tobe normal and an (N+1)th frame determined to be normal. Furthermore, theembodiment is characterized in that the image-data correction unit 107replaces an Nth frame determined to be abnormal with an (N−1)th framedetermined to be normal (step S402).

Other Embodiments

The present invention can be applied to a system constituted by aplurality of devices (e.g., a host computer, interface, reader, printer,etc.) or to an apparatus comprising a single device (e.g., a copier orfacsimile machine, etc.).

Further, it goes without saying that the object of the invention isattained also by supplying a recording medium (or storage medium) onwhich the program codes of the software for performing the functions ofthe foregoing embodiments to a system or an apparatus have beenrecorded, reading the program codes with a computer (e.g., a CPU or MPU)of the system or apparatus from the recording medium, and then executingthe program codes. In this case, the program codes read from therecording medium themselves implement the novel functions of theembodiments, and the program codes per se and recording medium storingthe program codes constitute the invention. Further, besides the casewhere the aforesaid functions according to the embodiments areimplemented by executing the program codes read by a computer, it goeswithout saying that the present invention covers a case where anoperating system or the like running on the computer performs a part ofor the entire process based upon the designation of program codes andimplements the functions according to the embodiments.

It goes without saying that the present invention further covers a casewhere, after the program codes read from the recording medium arewritten in a function expansion card inserted into the computer or in amemory provided in a function expansion unit connected to the computer,a CPU or the like contained in the function expansion card or functionexpansion unit performs a part of or the entire process based upon thedesignation of program codes and implements the function of the aboveembodiment.

In a case where the present invention is applied to the above-mentionedrecording medium, program code corresponding to the flowcharts describedearlier is stored on the recording medium.

Thus, in accordance with the present invention, as described above, evenif shape data representing the shape of an object in a moving imageundergoes a major change temporarily, a frame that is to be correctedcan be detected and corrected in ideal fashion and it is possible toexecute moving-image coding that is outstanding both visually and interms of coding efficiency.

The present invention is not limited to the above embodiments andvarious changes and modifications can be made within the spirit andscope of the present invention. Therefore, to apprise the public of thescope of the present invention, the following claims are made.

1. An image processing apparatus, comprising: moving-image input meansfor inputting a moving image composed of a plurality of frames;background-image acquisition means for acquiring a background imagerelating to the moving image that has been input; object extractionmeans for extracting an object by comparing each of the framesconstituting the moving image with the background image; abnormal-datadiscrimination means for discriminating whether shape data representingthe shape of the extracted object is abnormal or not; shape-datacorrection means for correcting the shape data based upon a result ofdiscrimination of the shape data; image-data correction means forgenerating image data, which represents the image of the object,conforming to the shape data that has been corrected by said shape-datacorrection means; and coding means for coding the shape data and theimage data, wherein said abnormal-data discrimination meansdiscriminates the necessity of correcting the shape data by comparingthe shape data of the object from frame to frame of a plurality offrames that differ in time, and wherein said abnormal-datadiscrimination means comprises: first comparison means for comparing theshape data of the object using a present frame and a preceding frame;second comparison means for comparing the shape data of the object usingthe preceding frame and a succeeding frame; and decision means fordeciding that correction of the present frame is necessary if adifference in the shape data of the object between the present frame andthe preceding frame is large and a difference in the shape data of theobject between the preceding frame and the succeeding frame is small. 2.The apparatus according to claim 1, wherein said first or secondcomparison means performs the comparison using a comparison of any ofthe area, perimeter, wavelet identifier, circularity, centroid or momentof the shape data, or a combination thereof.
 3. An image processingapparatus, comprising: moving-image input means for inputting a movingimage composed of a plurality of frames; background-image acquisitionmeans for acquiring a background image relating to the moving image thathas been input; object extraction means for extracting an object bycomparing each of the frames constituting the moving image with thebackground image; abnormal-data discrimination means for discriminatingwhether shape data representing the shape of the extracted object isabnormal or not; shape-data correction means for correcting the shapedata based upon a result of discrimination of the shape data; image-datacorrection means for generating image data, which represents the imageof the object, conforming to the shape data that has been corrected bysaid shape-data correction means; and coding means for coding the shapedata and the image data, wherein said abnormal-data discrimination meansdiscriminates the necessity of correcting the shape data by comparingthe shape data of the object from frame to frame of a plurality offrames that differ in time, and wherein said abnormal-datadiscrimination means comprises: first comparison means for comparing theshape data of the object using a present frame and a preceding frame;second comparison means for comparing the shape data of the object usingthe present frame and a succeeding frame; and decision means fordeciding that correction of the present frame is necessary if adifference in the shape data of the object between the present frame andthe preceding frame is large and a difference in the shape data of theobject between the present frame and the succeeding frame is large. 4.An image processing apparatus, comprising: moving-image input means forinputting a moving image composed of a plurality of frames;background-image acquisition means for acquiring a background imagerelating to the moving image that has been input; object extractionmeans for extracting an object by comparing each of the framesconstituting the moving image with the background image; abnormal-datadiscrimination means for discriminating whether shape data representingthe shape of the extracted object is abnormal or not; shape-datacorrection means for correcting the shape data based upon a result ofdiscrimination of the shape data; image-data correction means forgenerating image data, which represents the image of the object,conforming to the shape data that has been corrected by said shape-datacorrection means; and coding means for coding the shape data and theimage data, wherein said abnormal-data discrimination meansdiscriminates the necessity of correcting the shape data by comparingthe shape data of the object from frame to frame of a plurality offrames that differ in time, and wherein said abnormal-datadiscrimination means comprises: first comparison means for comparing theshape data of the object using a present frame and a preceding frame;second comparison means for comparing the shape data of the object usingframes following the present frame and the preceding frame; and decisionmeans for deciding that correction of prescribed frames from the presentframe onward is necessary if a difference between the present frame andthe preceding frame is large and a difference in the shape data of theobject between the preceding frame and the frames following the presentframe is small.
 5. An image processing apparatus, comprising:moving-image input means for inputting a moving image composed of aplurality of frames; background-image acquisition means for acquiring abackground image relating to the moving image that has been input;object extraction means for extracting an object by comparing each ofthe frames constituting the moving image with the background image;abnormal-data discrimination means for discriminating whether shape datarepresenting the shape of the extracted object is abnormal or not;shape-data correction means for correcting the shape data based upon aresult of discrimination of the shape data; image-data correction meansfor generating image data, which represents the image of the object,conforming to the shape data that has been corrected by said shape-datacorrection means; and coding means for coding the shape data and theimage data, wherein said shape-data correction means corrects the shapedata of the object in an Nth frame, which has been determined to beabnormal, using an (N−1)th or earlier frame determined to be normal andan (N+1)th or later frame determined to be normal, and wherein saidshape-data correction means comprises detection means for detectingcorresponding points in the shape data between an (N−1)th or earlierframe and an (N+1)th or later frame.
 6. The apparatus according to claim5, wherein said detection means obtains amount of motion of thecorresponding points by performing pattern matching with respect to oneor each of a plurality of prescribed areas.
 7. The apparatus accordingto claim 6, wherein the amount of motion of the corresponding points isan amount of movement and an amount of rotation of the position of thecorresponding points obtained by an affine transformation.
 8. An imageprocessing method, comprising: a moving-image acquisition step ofacquiring a moving image composed of a plurality of frames; abackground-image acquisition step of acquiring a background imagerelating to the moving image; an object extraction step of extracting anobject by comparing each of the frames constituting the moving imagewith the background image; an abnormal-data discrimination step ofdiscriminating whether shape data representing the shape of theextracted object is abnormal or not; a shape-data correction step ofcorrecting the shape data based upon a result of discrimination of theshape data; an image-data correction step of generating image data,which represents the image of the object, conforming to the shape datathat has been corrected in said shape-data correction step; and a codingstep of coding the shape data and the image data, wherein saidabnormal-data discrimination step includes discriminating the necessityof correcting the shape data by comparing the shape data of the objectfrom frame to frame of a plurality of frames that differ in time, andwherein said abnormal-data discrimination step further includes: a firstcomparison step of comparing the shape data of the object using apresent frame and a preceding frame; a second comparison step ofcomparing the shape data of the object using the preceding frame and asucceeding frame; and a decision step of deciding that correction of thepresent frame is necessary if a difference in the shape data of theobject between the present frame and the preceding frame is large and adifference in the shape data of the object between the preceding frameand the succeeding frame is small.
 9. The method according to claim 8,wherein said first or second comparison step performs the comparisonusing a comparison of any of the area, perimeter, wavelet identifier,circularity, centroid or moment of the shape data, or a combinationthereof.
 10. An image processing method, comprising: a moving-imageacquisition step of acquiring a moving image composed of a plurality offrames; a background-image acquisition step of acquiring a backgroundimage relating to the moving image; an object extraction step ofextracting an object by comparing each of the frames constituting themoving image with the background image; an abnormal-data discriminationstep of discriminating whether shape data representing the shape of theextracted object is abnormal or not; a shape-data correction step ofcorrecting the shape data based upon a result of discrimination of theshape data; an image-data correction step of generating image data,which represents the image of the object, conforming to the shape datathat has been corrected in said shape-data correction step; and a codingstep of coding the shape data and the image data, wherein saidabnormal-data discrimination step includes discriminating the necessityof correcting the shape data by comparing the shape data of the objectfrom frame to frame of a plurality of frames that differ in time, andwherein said abnormal-data discrimination step further includes: a firstcomparison step of comparing the shape data of the object using apresent frame and a preceding frame; a second comparison step ofcomparing the shape data of the object using the present frame and asucceeding frame; and a decision step of deciding that correction of thepresent frame is necessary if a difference in the shape data of theobject between the present frame and the preceding frame is large and adifference in the shape data of the object between the present frame andthe succeeding frame is large.
 11. An image processing method,comprising: a moving-image acquisition step of acquiring a moving imagecomposed of a plurality of frames; a background-image acquisition stepof acquiring a background image relating to the moving image; an objectextraction step of extracting an object by comparing each of the framesconstituting the moving image with the background image; anabnormal-data discrimination step of discriminating whether shape datarepresenting the shape of the extracted object is abnormal or not; ashape-data correction step of correcting the shape data based upon aresult of discrimination of the shape data; an image-data correctionstep of generating image data, which represents the image of the object,conforming to the shape data that has been corrected in said shape-datacorrection step; and a coding step of coding the shape data and theimage data, wherein said abnormal-data discrimination step includesdiscriminating the necessity of correcting the shape data by comparingthe shape data of the object from frame to frame of a plurality offrames that differ in time, and wherein said abnormal-datadiscrimination step further includes: a first comparison step ofcomparing the shape data of the object using a present frame and apreceding frame; a second comparison step of comparing the shape data ofthe object using frames following the present frame and the precedingframe; and a decision step of deciding that correction of prescribedframes from the present frame onward is necessary in a case where adifference between the present frame and the preceding frame is largeand a difference in the shape data of the object between the precedingframe and the frames following the present frame is small.
 12. An imageprocessing method, comprising: a moving-image acquisition step ofacquiring a moving image composed of a plurality of frames; abackground-image acquisition step of acquiring a background imagerelating to the moving image; an object extraction step of extracting anobject by comparing each of the frames constituting the moving imagewith the background image; an abnormal-data discrimination step ofdiscriminating whether shape data representing the shape of theextracted object is abnormal or not; a shape-data correction step ofcorrecting the shape data based upon result of discrimination of theshape data; an image-data correction step of generating image data,which represents the image of the object, conforming to the shape datathat has been corrected in said shape-data correction step; and a codingstep of coding the shape data and the image data, wherein saidshape-data correction step includes correcting the shape data of theobject in an Nth frame, which has been determined to be abnormal, usingan (N−1)th or earlier frame determined to be normal and an (N+1)th orlater frame determined to be normal, and wherein said shape-datacorrection step further includes a detection step of detectingcorresponding points in the shape data between an (N−1)th or earlierframe and an (N+1)th or later frame.
 13. The method according to claim12, wherein said detection step includes obtaining an amount of motionof the corresponding points by performing pattern matching with respectto one or each of a plurality of prescribed areas.
 14. The methodaccording to claim 13, wherein the amount of motion of the correspondingpoints is an amount of movement and an amount of rotation of theposition of the corresponding points obtained by an affinetransformation.
 15. A computer-readable storage medium storing a programwhich, when executed, performs a method for processing an image, theprogram comprising: code for a moving-image input step of inputting amoving image composed of a plurality of frames; code for abackground-image acquisition step of acquiring a background imagerelating to the moving image that has been input; code for an objectextraction step of extracting an object by comparing each of the framesconstituting the moving image with the background image; code for anabnormal-data discrimination step of discriminating whether shape datarepresenting the shape of the extracted object is abnormal or not; codefor a shape-data correction step of correcting the shape data based upona result of discrimination of the shape data; code for an image-datacorrection step of generating image data, which represents the image ofthe object, conforming to the shape data that has been corrected in theshape-data correction step; and code for a coding step of coding theshape data and the image data, wherein said abnormal-data discriminationstep includes discriminating the necessity of correcting the shape databy comparing the shape data of the object from frame to frame of aplurality of frames that differ in time, and wherein said abnormal-datadiscrimination step further includes: a first comparison step ofcomparing the shape data of the object using a present frame and apreceding frame; a second comparison step of comparing the shape data ofthe object using the preceding frame and a succeeding frame; and adecision step of deciding that correction of the present frame isnecessary if a difference in the shape data of the object between thepresent frame and the preceding frame is large and a difference in theshape data of the object between the preceding frame and the succeedingframe is small.
 16. A computer-readable storage medium storing a programwhich, when executed, performs a method for processing an image, theprogram comprising: code for a moving-image input step of inputting amoving image composed of a plurality of frames; code for abackground-image acquisition step of acquiring a background imagerelating to the moving image that has been input; code for an objectextraction step of extracting an object by comparing each of the framesconstituting the moving image with the background image; code for anabnormal-data discrimination step of discriminating whether shape datarepresenting the shape of the extracted object is abnormal or not; codefor a shape-data correction step of correcting the shape data based upona result of discrimination of the shape data; code for an image-datacorrection step of generating image data, which represents the image ofthe object, conforming to the shape data that has been corrected in theshape-data correction step; and code for a coding step of coding theshape data and the image data, wherein the abnormal-data discriminationstep includes discriminating the necessity of correcting the shape databy comparing the shape data of the object from frame to frame of aplurality of frames that differ in time, and wherein the abnormal-datadiscrimination step further includes: a first comparison step ofcomparing the shape data of the object using a present frame and apreceding frame; a second comparison step of comparing the shape data ofthe object using the present frame and a succeeding frame; and adecision step of deciding that correction of the present frame isnecessary if a difference in the shape data of the object between thepresent frame and the preceding frame is large and a difference in theshape data of the object between the present frame and the succeedingframe is large.
 17. A computer-readable storage medium storing a programwhich, when executed, performs a method for processing an image, theprogram comprising: code for a moving-image input step of inputting amoving image composed of a plurality of frames; code for abackground-image acquisition step of acquiring a background imagerelating to the moving image that has been input; code for an objectextraction step of extracting an object by comparing each of the framesconstituting the moving image with the background image; code for anabnormal-data discrimination step of discriminating whether shape datarepresenting the shape of the extracted object is abnormal or not; codefor a shape-data correction step of correcting the shape data based upona result of discrimination of the shape data; code for an image-datacorrection step of generating image data, which represents the image ofthe object, conforming to the shape data that has been corrected in theshape-data correction step; and code for a coding step of coding theshape data and the image data, wherein the abnormal-data discriminationstep includes discriminating the necessity of correcting the shape databy comparing the shape data of the object from frame to frame of aplurality of frames that differ in time, and wherein the abnormal-datadiscrimination step further includes: a first comparison step ofcomparing the shape data of the object using a present frame and apreceding frame; a second comparison step of comparing the shape data ofthe object using frames following the present frame and the precedingframe; and a decision step of deciding that correction of prescribedframes from the present frame onward is necessary if a differencebetween the present frame and the preceding frame is large and adifference in the shape data of the object between the preceding frameand the frames following the present frame is small.
 18. Acomputer-readable storage medium storing a program which, when executed,performs a method for processing an image, the program comprising: codefor a moving-image input step of inputting a moving image composed of aplurality of frames; code for a background-image acquisition step ofacquiring a background image relating to the moving image that has beeninput; code for an object extraction step of extracting an object bycomparing each of the frames constituting the moving image with thebackground image; code for an abnormal-data discrimination step ofdiscriminating whether shape data representing the shape of theextracted object is abnormal or not; code for a shape-data correctionstep of correcting the shape data based upon a result of discriminationof the shape data; code for an image-data correction step of generatingimage data, which represents the image of the object, conforming to theshape data that has been corrected in the shape-data correction step;and code for a coding step of coding the shape data and the image data,wherein the shape-data correction step includes correcting the shapedata of the object in an Nth frame, which has been determined to beabnormal, using an (N−1)th or earlier frame determined to be normal andan (N+1)th or later frame determined to be normal, and wherein theshape-data correction means further includes a detection step ofdetecting corresponding points in the shape data between an (N−1)th orearlier frame and an (N+1)th or later frame.